Don't Get Cut on Occam's Razor!

On Occam's Razor

\Occam's Razor" is an important metatheoretical principle in science. Mathematical methods from ergodic theory, information theory, and the theory of algorithms and stochastic processes can be used to help clarify the meaning of Occam's Razor and address the philosophical problems it raises. 1 1 \Pluralitas non est ponenda sine nec-cesitate." Thus, William of Occam articulated the famous princi...

A note on Occam’s razor

(a) (b) Figure 1: (a): The likelihood p(D|θ,Mcomplex) has a higher maximum value than then simpler model, but the likelihood drops quickly as we move away from regions of high likelihood. (b): The likelihood p(D|θ,Msimple) has a lower maximum value than then complex model, but the likelihood changes less quickly as we move away from regions of high likelihood. The corresponding fraction of para...

How to Cut Costs — and Get Your Employees to Help

Occam's Razor

The Bayesian paradigm apparently only sometimes gives rise to Occam’s Razor; at other times very large models perform well. We give simple examples of both kinds of behaviour. The two views are reconciled when measuring complexity of functions, rather than of the machinery used to implement them. We analyze the complexity of functions for some linear in the parameter models that are equivalent ...

Sharpening Occam's Razor

We provide a new representation-independent formulation of Occam’s razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based [4] and VC-based [3] versions of Occam’s razor theorem, in many applications; and (ii) Achieve a sharper reverse of Occam’s razor theorem than that of [5]. Specifically, we weaken the assum...